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DESCRIPTION OF PLANS FOR ROCKEFELLER FELLOWSHIP 


BASIC DESIGN GOALS 


ihe primary design goal of the Digital Image Processor (DIP) is to provide a comgre- 
hensive video image processing device for. the -individual video worker, that is to 

Say, to provide in a single instrument all ofthe control that a video worker requires 
to create or, as some say, produce work. This would include standard special effects 


that a TV studio can produce like fades and Switches, color correction for errors 


in illumination, noise reduction and image enhancement techniques, time base 
correction, squeeze zooms, multiple images, computer control of tape recorders for 
editing, flip-flops, zaps, and, of course, color chromakey. In addition to effects 
that have names, presently used in Studios, it would be able to impiement an immense 
variety of processes and patterns that the user.will be able to create. The basic 
concept here is the creation of a generalized device rather than a special purpose 
device. A modern TV studio uses the organizing principle of a large number cof 
special purpose devices which are interconnécted to produce the required processins 
for the TV signal. The Digital Image Processor is a general purpose device which 
can implement a larger range of processing than combinations of special purpose 
devices in a color TV studio. . 


The second major design goal is a user oriented or habitable system. It is clear 
that a general purpose device, a device that can do an immense variety of things, 

is inherently more difficult to control than a set of special purpose devices. 

Great pains will be taken to make the DIP accessible and easy to use by peopie without 
a large amount of technical training. Critical to the implementation of this aesica 
goal is the use of a small computer graphics system, the Z-Grass macr” ie developed 

by Dr. Thomas DeFanti.,to control the DIP. The primary function of this computer 
graphics system will be to accept control information from the user asc to inform 

the user what the state of the instrument js. The’ input control can come in many 
forms. It can be commands on a typewriter keyboard, dial turning, joy stick one gia me | See 
various touch sensitive devices, or even sensitive Clothing. The concept here is - 
that the general purpose digital computer forms an intelligent interface between the 
control devices which are run by the user of the instrument and the actual control 

of the Digital Image Processor. It jis hoped that this will accomplish the cvals 

of both a highly facile performance instrument in the sense of a musica] per forminc 
Instrument and a program controllable studio instrument. 


HISTORY, PRESENT AND FUTURE OF THE PROJECT 


I began to consider building this instrument about five years ago. Two years adu. 
I received a Guggenheim Fellowship to design and construct what turns out to be the 
conceptual father of this design. The difficulty was that cae desicn was of such 
large scale and of such expense that near the end of the fellowshin neriod whan 

I had completed a great deal of the design and had actually begun corstruct.un of 
one of the modules, it became clear that I couldn't continue without jterqe scale 
industrial support and a veritable research 1abOratery O07 €, eCtronte desi aiers. 

In virtual despair over this Situation, the idea of time division Multiplexing 

the signal and lowering the number of bits necessary to represent the siaqnal 
occurred to me. This ephemeralized the design to between one-third ond one-ninth 
of its previous size. This brought the design back into a cost rance that would 
be affordable by dedicated individuals and by small institutions. Fortunately, 
only about a third of the work completed on the Guggenheii Fellowship 4s lost. 

The rest applies rather directly to this design, including a larce (arts stock: 

















t this point, I am ready to start construction of the first modules for the new 
Digital Image Processor (DIP). Financed by my own money, progress will be very 
Slow. The major virtues for the video community of supporting this work with a 
Rockefeller Fellowship would be as follows: 


The instrument would be completed years before it would be under my own individual 
resources. The free distribution of the plans to individuals and not-for-profit 
Institutions would be assured and the reduction to printed circuit board instead 

of wirewrap technology would make the instrument. twice as compact, which really 
means twice as powerful in the same physical size, and would greatly ease the effort 
of copying the instrument. 1 would not be able to afford this design step, the 
reduction from wirewrap to printed circuit board with my own resources. 


DISTRIBUTION 


My plans for distribution of this design, should I receive a Rockefeller Fellowship, 
would be as follows: 


I will complete sufficient documentation of the instrument so that an individual 
without immense technical background would be able to successfully copy the instru- 
ment and then I will give away the plans for the cost of postage and printing 

cost or some nominal charge associated with my cost of distributing them. 


This distribution technique may seem 1]1]1-conceived at first look, but in the case 
of the Analog Image Processor, it turned out to be an extremely effective form of 
aTStri bution. First of @lys< 4t. hae resulted In over 20 copies of the Analog Image 
Processor (AIP) being constructed, mostly by individual artists who actually 

use the instrument to'do a large range of productions. A considerable percentage | 
of video art broadcast in Chicago has been performed on the AIP and there is a 
large number of artists who would be doing an entirely different kind of expressive 
work without the availability of this instrument at low financial cost. An example 


of another distribution technique would be to try and manufacture and sell the 
Instrument. A comparable instrument in Structure to the AIP is the Rutt-Etra 


Scan Processing Video Synthesizer. It turns out that there are approximately 

equal numbers of the Rutt-Etra Synthesizer and the AIP indicating rough parity in 
terms of success of the two distribution techniques. The AIP, however, is on its 
own. It is still being copied and constructed by individuals with relatively littte 


effort on my own part, so that it is likely that the number of Image Processors 
will continue to grow in the near future. | 


A second aspect of this distribution technique is that because it requires a relatively 
Tow financial commitment and a relatively large personal time commitment, it tends 

to lend itself to the individual practicing video worker, who typically has rela- 
tively small income but considerable dedication, so I consider it to be an ootimum 
distribution technique for the designs of the DIP. 
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BUDGET SUMMARY ' 


The $10,000 stipend, I would use to take a leave of absence from th 


ie University 
of Illinois at Chicaao Circle for between 6 .and 9 months to focus on the imple- 


mentation and documentation of the design of the DIP. What Follows is an 
approximate budget for the $25,000 expense category. 


Test Equipment | $3,000 
Electronic and Printed Circuit Design Services 25000 
Printed Circuit Production Services . | 4,000 
Z-Grass Computer _ 2 5000 
Specialized Software for Computer 9 5000 
Electronic Parts : | 3.000: 
Help on Documentation 14000 
Printing of Documentation | 1,000 
Telecommunications 3 750 


iliscellaneous. Office Supplies, Videotape, 
Digital Storage Medium 1,000 


Travel (My collaborators On TATS. DROJECE. [Ve 
on the West Coast.) 3000 


Administrative Expense (probably scarfed up by 


the sponsoring institution) at least 3,006 


This totals to more than $25,000. The difference plus probably much more will 
come from my personal Savings. 
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TECHNICAL DESCRIPTION, OF INSTRUMENT 

Figure 1 is the first systems block diagram. As indicated above, there is a 
general purpose computer graphics system which controls a special purpose video 
Signal processor called the Digital Image Processor and also receives input from 
conventional computer LerminadsS. dials2 joysticks, pressure sensitive surfaces, 
keyboards, and a variety of input devices, many of which I have already developed 


under a National Endowment for the Arts grant. The report of that Grant is 
included in the Appendices. | 


The communication Protocol] used between the general purpose computer and the digital 
Signal processor is a Standard parallel I-0 port configuration, so that as new 
developments in low cost computing become avatlable, they can be Incorporated for 
controlling digital image processors. The computer chosen for development is the 
Z-Grass machine. The ‘hardware was develoned by. Dave Nutting and Associates and 

it will be manufactured and distributed by Datamax Corporation. It is a 780 based 
computer graphics system which is exquisitely well adapted to the CONLYO) Of Wisya | 
and sonic entities. The software for that system was developed by Dr. Thomas DeFantj 
Of the University of TOS at Chicago Circle and represents the state of the art 
in interactive real-time control of images and events. Dr. VeFanti has committed 
himself to extensions to the Z-Grass language explicitly for the control of the 
Digital Image Processor (DIP). It is also likely that Terry Diz wil] complete 

a doctoral dissertation in Information Engineering on the inplementation of a user 
orjented contro] system for the Digital Image Processor in Z-Grass. It 1S most 


fortunate that these two highly skilled and creative workers are able to contribute 
to this project. 


The DIP will be constructed in four successive phases. Tne block Giagrams |] carough 
8 represent the Phase 1 instrument. As indicated in Figure 1, the digital Srqnel 
Processor contains two main subsections, the analog box and the digital box. Figure 

é 1S a block diagram to card level of the analog box. The analog box takes care of 
all of the video Protocol. It genlocks to an externa} source, does the color cecoding 
and analog to digital conversion for color cameras and the digital to analog 
conversion for output. Figure 3 is a diagram to card level of the digital box. The 
digital box contains the diaital processing modules, an 8x8 matrix TOY interconnecting 
the digital processing moduels, a digital color encoder, two pattern generation 
modules and a test generator and receiver module. Figure 4 shows the interconnectisn 
diagram of the processing modules, camera and pattern generator inputs and the 2x8 
matrix. The matrix allows connection from any input on the left to any set of outsauts 
on the bottom. The digital signals routed by that matrix to the Processing modules 
are parallel pipeline time division multiplexed Signals. Parallel] Pipeline architecture 
1S Similar to a bucket brigade. Figure 5 is a diagram illustrating this. The Signal 
coming from the left which, say, represents a gray value, is latched at the first 
latch, processed and then passed on to the next Stage where it is latched again. At 
the same time a new piece of information is coming in from the left. From the middle 
latch the information 4s processed and sent ot the next latch aéd on the next clock 
tick, that information 1S latched. So that on each clock tick, information 1S Stored. 


“processed, and transferred to the next latch. The instrument runs at J4.31818 


megahertz, meaning a 69 nanosecond processing time from Jatch to bates - This: as 3 
digital architecture that 1S capable of Passing and processina information Fast 

enough to do real-time video. The time division multiplexing means that on succeeding 
Clock ticks, different kinds of information are transferred. A color experience js 
characterized by three dimensions of Information, normally thought of as red, green, 
and blue, the amount of primaries associated with any color experience. Time division 
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multiplexing, for instance, could send a red Signal, then a green siqnal, then a biue 
Signal. But to take advantage of inherent encoding efficiencies of NTSC video, the 
time division multiplexing is as follows. ° A sample of Y luminence is sent and then 

a sample of R-Y, which is one of the chromance Signals, another sample of luminence, 
and then a sample of B-Y is sent, and then it repeats again with Y. In this forn 
Tuminence information or gray value information is sampled twice as often as the 


chromance information. This js consistent with the amount of information available 
In eahc of those categories with NTSC Ty. 


Another efficiency of the design is to send Y as 7 bits worth of information, meaning 
128 possible values and to send R-Y and B-Y with 6 bits worth of information or 


64 possibilities each. This is also consistent with the amount of information 
available in NTSC TV. 


The basic processing modules are Tllustrated in Figure 6. The physical module is 
enclosed in a dotted line, but it is best to think of the process as the logical 
module illustrated. The lookup table outside of the dotted lines js actually 
present in the next physical module. The lookup tables illustrated are high speed 
memories where the data coming in is the address and the data GOING: Out. Ts: the. data 
Stored at that address. This forms an arbitrary lookup table so that for every value 
coming in there is an arbitrary value going out. Because of the time division 
multiplexing, each box can be thought of us three independent lookup tables, 

one for Y, one for R-Y, and one for B-Y. In addition to the lookup tables, there 
are delay elements which can generate an arbitrary delay of the signal. It 
essentially is a pipeline with no processing, and an adder indicated by a box with 
a+ sign. This form of module is Capable of doing colorization, color correction, 
addition (which is superimposition), multiplication (which is keying), and utilizing 
the delay elements can form both transversal and recursive filters that are useful 
In controlling the amount of detai} present in the inage and can be used for various 
image enhancement and. noise reduction algorithms. There are 3 of these logical 
modules in the Phase 1 DIP. The pattern generators are illustrated in Figure 7. 
They are composed of phase accumulators, memory lookup tables, and adders. In 
addition to generating the classic wipe and split screen patterns and varying then 
dynamically, it can generate a very large range of interesting images, one of the 
more bizarre of which is illustrated in Figure 8. These patterns can be used as 
images themselves or can be used to control the processing of other images. 


The Phase 1 DIP is expected to cost about $5,000 in Daytss 16 addi 2167p to. that the 


computer system required to contro] it wili cost between $2,500 and $5,000 dependine 
On peripherals. The chief limitation of the Phase 1] instrument is that it will not 
be able to utilize more than one videotape input. In order to use more than cne 
Videotape input, multiple time base correctors are required. These can be added 


to the design for approximately $1000 a channel in parts. 


The Phase 2 instrument would be able to do A-B roll editing throuch an image processor 
and computer control of tape recorders forming a complete postproduction facility 
Capable of wipes and fades and keys along with more interesting combinations of two 

or more Input sources on tape. 


The Phase 3 DIP would include a new Structure called the convolution and geometric 
processing module. Since all the inputs and outputs of the matrix are already 
used in the Phase 1 and 2 Instrument, this would require enlarging the switching 
matrix and adding more mechanical boxes to the System. Tne convolution function 
is extremely powerful in doing advanced #mage enhancement, feature extractions 

and noise reduction techniques in addition to giving the user very detailed 
control over the amount of spatial information in the image beina processed. The 
geometric processing element of the convolutior processor would allow all of tne 
effects which are normally associated with a fram buffer such as Squeezing and 


«ihe * 


«» flipping and multiple Images, but only in the horizontal direction. In other words, 
you could create a bilaterally symmetric image or the same image repeated twice 
in the horizontal direction. The price of the convolution processor at this point 
is uncalculated. I quess it to be around 51900 a channel in parts. 


| The Phase 4 instrument would utlize a full frame buffer. This frame buffer would 

| be external to the instrument and probably af someone else's design. There are a 
number of excellent people in the country working on versatile frame buffers. 
With this element, one would be able to do functions that are euphemistically 


called electronic opticals, such as zoom-ins and zoom-outs (called Squeezes), pans, 
rotations, and multiple images. | 
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